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Acquisition  Support  Program  Mission 


Drawing  on  our  expertise  in  software  engineering,  help  DoD,  federal 
agency,  and  other  acquirers  continuously  improve  and  institutionalize  their 
ability  to  acquire,  deploy,  and  sustain  systems  and  capabilities. 

Identify  opportunities  for  the  Software  Engineering  Institute  (SEI)  to  create, 
apply,  and  amplify  technologies  that  respond  to  customer  needs. 

Disseminate  lessons  learned  and  best  practices  through  courses, 
workshops,  conferences,  publications,  and  participation  in  acquisition 
communities  of  practice. 
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Strategies 


Impact  individual  programs 
Impact  acquisition  organizations 
Define,  integrate,  and  transfer  knowledge 
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Sampling  of  Customers 


Army 

•  AMCG-6 

•  AMRDECSED 

•  ASA(ALT) 
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•  PEO  Aviation 
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Combat 
Systems 

•  PM  Integrated 
Fire  Control 


Navy 

•  Common  Link 
Integrated 
Processor 

•  DDG-1000 

•  F/1 8 

•  SPAWARPMW 
146-San  Diego 

•  SPAWAR  Systems 
Center-Charleston 

Civil  Agencies 

•  Dept,  of  Interior 

•  Dept.  Veterans 
Affairs 

•  Nuclear  Regulatory 
Commission 

•  NASA 

•  USCoastGuard 


Software  Engineering  Institute 
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ASP  Army  Team 

Chief  Engineer:  Cecilia  Albert 


Software  Engineering  Institute  CarnegieMellon 
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ASP  Army  Team  -  Strategy 


Impact  Acquisition  Orgs 

•  Work  across  the  Army  through  ASA/ALT  to  “dramatically  improve  the 
acquisition  of  software-intensive  systems” 

•  Work  with  selected  PEOs  to  implement  improvements  across  their 
organizations 

Impact  Individual  Programs 

•  Work  with  selected  PMs  to  help  them  be  more  effective  and 
demonstrate  the  utility  of  “best”  software  engineering  practices 

Define,  Integrate,  and  Transfer  Knowledge 

•  Help  Software  Engineering  Centers  become  transition  partners  for 
providing  software  engineering  services  to  Army  PMs 

•  Help  define  Army  needs  for  software  engineering  technology 
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Army  Example  Engagement  Results., 

Adaptive  Planning  Tool  Migration  to  NECC 


Task: 


•  Define  an  acceptable  transition  path  for  rapidly  transitioning  proven 
ACTD/JCTDs  into  operational  capability  through  NECC  as  the  program  of 
record 

Strategy: 

•  Use  Quality  Attribute  Workshop  to  define  architecturally  significant 
scenarios 

•  Use  the  Service-Oriented  Migration  and  Reuse  Technique  (SMART)  to 
evaluate  candidate  tools  in  the  context  of  NECC 

•  Define  a  “fast  track”  process  to  take  advantage  of  ACTD  capabilities  to 
rapidly  (and  cheaply?)  deliver  operational  capability  through  NECC 
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Army  Example  Engagement  Results2 

Adaptive  Planning  Tool  Migration  to  NECC 


Accomplishments: 

•  Worked  with  JFCOM  and  JCS/J-9  to  understand  operational  needs 

•  Worked  with  NECC  to  understand  critical  evidence  required  for  entrance 

•  Examined  8  individual  products  for  applicability 

•  Developed  a  high  level  functional  architecture  using  3  products 
Next  Steps: 

•  Continue  dialogue  with  customers  and  NECC 

•  Define  interoperability  requirements  across  the  3  products 

•  Support  selection/award  of  Prime  Contractor 

•  Define  an  architecture-based  approach  for  Adaptive  Planning 

•  Document  agreed  upon  “fast  track”  approach  to  migration  to  NECC 
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ASP  Navy  Team 

Chief  Engineer:  Rick  Barbour 


Software  Engineering  Institute  CarnegieMellon 
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ASP  Navy  Team  -  Strategy 


Identify  and  focus  on  DoN  priorities  (e.g.  Systems/Software  Engineering, 
Open  Architecture,  Software  Assurance) 

Continue/Expand  current  funding  with  existing  programs  (DDG  1000, 
CG(X),  PEO  IWS,  F-18,  JSF,  SIAP  JPO,  SPAWAR  Systems  Center 
Charleston  (SSC-C) ) 

Work  with  selected  PMs  and  PEOs  to  improve  their  acquisition  capabilities 

•  System  and  Software  Architecture 

•  Process  Improvement — System/Software  Engineering  and  Acquisition 

•  Systems  Interoperability 

•  DoD  5000  Acquisition  Documents  (SEP,  TEMP,  AS) 

Develop  opportunities  to  embed  technical  staff  in  Navy  acquisition 
programs  and  PEOs 

Develop  senior  Navy  management  advocacy  (ASN  RDA/RDA  CHENG, 
DASN  IWS,  DASN  RDT&E) 
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Navy  Example  Engagement  Results- 1 

Early/  Continuous  Support  to  DDG  1000—Zumwalt  Class  Destroyer 

Task: 

•  Technical  support  of  the  DDG  1000  Acquisition  activities 
addressing  evaluation  of  Design  Agents  (DA)  contractor’s 
approach,  deliverables,  plans,  strategies  &  concepts 

•  Identify,  evaluate  and  mitigate  software  acquisition  risks 

•  Emphasis  on  software  architecture  evaluation,  fault  tolerance,  real¬ 
time  operations,  resource  management,  COTS  integration, 
software  development  and  acquisition  process,  and  technology 
transition 

Strategy: 

•  Provide  direct,  ongoing  support  to  acquisition  program  office 

•  Address  early  acquisition  issues  for  Total  Ship  Computing  Environment  (TSCE) 


TSCE  has  passed  through  multiple  software  releases  and  is  nearing  a 

production  readiness  milestone. 


—  SEPG  2008:  ASP 

3^—  Software  Engineering  Institute  CarnegieMellon 

’  ©  2008  Carnegie  Mellon  University 


Navy  Example  Engagement  Results-  2 

Early/  Continuous  Support  to  DDG  1000-Zumwalt  Class  Destroyer 


Accomplishments: 

•  SEI  has  been  an  integral  part  of  this  program  for  the  past  8  years 

•  Continuous  support  of  the  TSCEI  Tech  Team  IPTs 

•  Worked  on  Software  Requirements  for  Release  5 

•  worked  with  Raytheon  architects  to  define  system  quality  objectives  in 
Release  6  vision 

•  Consistent  input  on  “Work  Instruction  Improvement”  (e.g.  Peer  Review 
procedures  ) 

•  Support  has  focused  on  architecture  technical  analysis,  fault  tolerance  practices 
and  improved  configuration  and  risk  management 

•  Configuration  Management  analysis/recommendations  supplied  and  accepted 

•  Continuing  technical  support  in  testing  and  acquisition  support  as  program  nears 
production 


—  SEPG  2008:  ASP 

3^—  Software  Engineering  Institute  CarnegieMellon 

’  ©  2008  Carnegie  Mellon  University 


Navy  Example  Engagement  Results-  3 

Acquisition  Strategy  Development  for  Joint  Strike  Fighter 


Task: 


•  Technical  support  of  JSF  Architectural  Integrity,  architectural  related 
documentation  and  Technical  Interchange  Meetings  and  IPTs 

•  Support  Identification  of  Risk  and  Technology  Refreshment  issues 

•  Provide  independent,  on-going,  software  engineering  advisory 
expertise  to  JPO 


Strategy: 

•  Provide  direct,  ongoing  support  to  acquisition  Joint  Program  Office 

•  Work  closely  with  JPO  and  contractor 


Architecture:  using  Quality  Attribute  definition  and  architecture  evaluation  practices  on  F-35  program 

“...the  [SEI  architecture  work]  benefits  the  F-35  Joint  Strike  Fighter  (JSF)  Program... in  evaluating  our 
extensive  system  architecture  (both  airborne  and  ground-based)" 

Mike  Bossert,  F-35  Air  System  Architect” 
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Navy  Example  Engagement  Results-  4 

Acquisition  Strategy  Development  for  Joint  Strike  Fighter 


Accomplishments: 

•  Completed  Technical  System  Architecture  Assessments 

•  Completed  Automated  Logistics  Information  System  (ALIS)  Requirements  Review 

•  Developed  and  published  Quality  Assessment  of  Software  -Intensive  System 
Architectures  (QUASAR)  Method  CMU/SEI-2006-HB-001 

•  Developed  Periodic  Architecture  Assessment  Reporting  System  and  Evaluation 
(PARSE)  status  methodology  in  use  on  program  today 

•  Developed  JSF  specific  Training  materials  for: 

.  QUASAR 

•  Modular  Open  Systems  Approach  (MOSA)  and  associated  tool 

•  Open  Architecture  and  associated  Open  Architecture  Assessment  Tool 

•  Numerous  reviews  of  JPO  documents  (e.g.)  Requirements,  Plans,  Security 
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ASP  Air  Force  Team 


Chief  Engineer:  John  Foreman 


Software  Engineering  Institute  CarnegieMellon 
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ASP  Air  Force  Team  -  Strategy.) 


Impact  Individual  Programs 

•  Work  directly  with  key  acquisition  programs  to  help  them  achieve  their 
objectives.  Emphasis  on  major  acquisitions  with  PEO-based/product  center 
focus. 

—  Currently:  Space  (SMC)  and  Command  and  Control  (ESC) 

—  Future:  aircraft  and  weapons 
Impact  Acquisition  Organizations 

•  Maintain  support  from  the  Office  of  the  Assistant  Secretary  of  the  Air  Force 
for  Acquisition  (SAF/AQ)  through  work  planned  and  executed  in  direct 
support  of  the  Senior  Acquisition  Executive  (SAE)  to  expedite  the  rapid 
transition  of  SEI-recommended  best  practices 
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ASP  Air  Force  Team  -  Strategy2 


Define,  Integrate,  and  Transfer  Knowledge 

•  Disseminate  lessons  learned  and  best  practices  through  courses, 
workshops,  publications,  and  participation  in  acquisition  communities  of 
practice 

•  Look  for  and  leverage  commonality  for  the  benefit  of  others 
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ASP  Air  Force  Example  Engagement  Results., 

Software  Strategy  and  Expert  Advice  for  TSAT 


Task: 

•  Augmenting  and  enhancing  the  software  engineering  aspects  of  the 
program 

•  Providing  expert  advice  and  guidance  on  software  engineering 
management,  acquisition  and  technology 

Strategy: 

•  Documented  the  way  forward  for  TSAT  in  the  “TSAT  Recommended 
Software  Strategy” 

Accomplishments: 

•  Software  IPT  was  reconstituted 

•  SWAMP  has  been  written  and  approved 

•  System  level  software  architecture  is  being  developed  based  on  segment 
data  to  help  identify  software  risks  and  gaps  across  the  system 


—  SEPG  2008:  ASP 

3^—  Software  Engineering  Institute  CarnegieMellon 

’  ©  2008  Carnegie  Mellon  University 


ASP  Air  Force  Example  Engagement  Results2 

Software  Strategy  and  Expert  Advice  for  TSAT 


Customer  Response: 

•  According  to  Col  Moody: 

" The  SEI  team  has  truly  been  the  conscience  of  software  on  the  TSAT 
program.  The  Institute  has  provided  detailed  expertise  in  individual 
program  segments,  and  is  now  helping  the  program  with  tools  to  avoid 
software  pitfalls  within  the  system 

“From  them,  I  get  unflinching  views  from  people  who  have  seen  the  best  and 
worst  of  software  practices  on  large  software-intensive  programs. " 
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ASP  Intelligence 
Community  Team 

Chief  Engineer:  Rita  Creel 
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ASP  Intelligence  Community  Team  -  Strategy 


Assist  organizations  throughout  the  Intelligence  Community  in  improving  outcomes  of 
software-intensive  systems  acquisitions 

•  Directly  support  acquisition  program  offices,  identifying  risks  and  issues,  recommending 
solutions,  and  providing  needed  expertise 

•  Work  with  agency-  and  directorate-level  systems  engineering  and  acquisition  organizations 
to  improve  agency-wide  policies  and  guidance 

•  Collaborate  with  the  Office  of  the  Director  of  National  Intelligence  (ODNI)  on  DNI  policies 
affecting  software 

Establish  an  1C  “Software  Acquisition  Community  of  Practice” 

•  Encourage  collaboration  on  shared  software  concerns 

•  Define,  integrate,  and  transfer  knowledge 

Enhance  SEI  ability  to  support  Sensitive  Compartmented  Information  (SCI)  Programs 

•  Improve  security  practices  and  procedures 

•  Obtain  sponsorship  for  more  clearances 

•  Establish  an  SCI  processing  infrastructure 
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Intelligence  Agencies  -  Themes 


Educating  the  acquirer 

Imparting  requisite  software  knowledge  to  define,  monitor,  and  manage  a  program;  training 
and  mentoring;  effective  teaming. 

Advancing  software-aware  system  engineering 

Advising  on  requirements  engineering  and  management;  system  architecting,  design, 
construction,  and  integration;  verification  and  validation;  and  sustainment  and  refresh 
techniques  that  suit  complex  environments. 

Facilitating  horizontal  integration 

Guiding  the  acquirer  on  development  of  robust  architectures,  interoperable  systems, 
integration  of  disparate  data,  data  mining,  integrating  the  “enterprise,”  etc. 

Overcoming  process  aversion 

Communicating  the  value  of  process,  modeling  processes  to  identify  inefficiencies  and  the 
need  for  improvement. 

Overcoming  technology  aversion 

Understanding  prevalent  attitudes,  ensuring  people  are  considered  in  technology  solutions. 

Tempering  technology  worship 

Performing  robust  risk-benefit  analyses,  defining  feasible  off-ramps. 
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ASP  Civil/Defense 
Agency  Team 

Chief  Engineer:  Steve  Palmquist 
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ASP  C&DA  Team  -  Strategy 


Help  civil  and  defense  agencies  acquire  and  sustain  software-intensive 
systems  with  predictable  cost,  schedule,  and  quality  by 

•  Engaging  with  customers  whose  issues  align  with  the  goals  of  the  SEI’s 
technical  initiatives 

•  Helping  customers  solve  their  current  problems 

•  Growing  the  customer’s  ability  to  solve  their  future  problems 

•  Reaching  out  to  the  larger  software  acquisition  and  development 
community  through  presentations,  technical  reports,  and  workshops 
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Civil  &  Defense  Example  Engagement  Results 

IRS  Customer  Account  Data  Engine  (CADE) 


Task: 

•  Initially  asked  by  the  program  office  to  look  at  the  use  of  a  business  rules 
engine  (BRE)  in  CADE 

•  IRS  Commissioner  intervened  to  expand  to  a  full  Independent  Technical 
Assessment  (ITA)  after  the  prime  contractor  missed  a  major  deliverable 

Strategy: 

•  In  Congressional  testimony,  SRS  Commissioner  stated  the  SEI  would 
assess  the  CADE  program  yearly,  which  we  have  supported  (2003,  2004, 
2006,  &  2007;  2008  in  planning) 

•  SEI  is  one  of  several  organizations  assessing  CADE  (Treasury  IG,  IRS 
Congressional  Oversight) 

Accomplishments: 

•  CADE  program  has  met  significantly  more  cost,  schedule,  and  performance 
targets 
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Knowledge  Integration 
and  Transfer 


Software  Engineering  Institute  CarnegieMellon 
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Why  is  Software-Intensive  Acquisition  Hard? 


Complex  interactions  between  PMO,  contractors,  sponsors,  and  users 

•  Full  chain  of  actions  &  their  longer-term  consequences  are  not  clear 

•  Hard  to  apply  corrective  actions  when  status  is  uncertain 
Significant  delays  exist  between  applying  changes  and  seeing  results 

•  Difficult  to  control  systems  with  long  delays  between  cause  &  effect 

•  Example :  Steering  an  aircraft  carrier 

Unpredictable  and  unmanageable  progress  and  results 

•  Limited  visibility  into  real  progress  &  status 

•  Complexity  of  interdependencies  has  unintended  consequences 

Uncontrolled  escalation  of  situations  despite  best  management  efforts 

•  Misaligned  goals  can  drive  potentially  conflicting  behaviors 

Linear  partitioning  is  the  standard  approach  to  address  large  systems 

•  When  systems  have  feedback  between  components  that  are  partitioned,  it 
makes  it  difficult  to  see  &  address  these  interactions 

Exponential  growth  of  interactions  as  size  grows  linearly 
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Acquisition  Archetypes 


ASP  is  producing  a  set  of 
‘Acquisition  Archetypes ” 
concept  briefs,  analyzing 
recurring  patterns  in 
actual  acquisition 
programs,  and 
recommending 
interventions  and 
preventative  actions 
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What  are  the  Acquisition  Archetypes? 


The  Acquisition  Archetypes  depict  the  underlying  structures  of  a  set  of 
dynamic  behaviors  that  occur  throughout  acquisition  organizations 

•  Each  diagram  tells  a  familiar,  recurring  story 

•  Each  describes  the  structure  that  causes  the  dynamic 
Acquisition  Archetypes  are  used  to: 

•  Identify  failure  patterns  as  they  develop  (recognition) 

•  Single  out  root  causes  (diagnosis) 

•  Engage  in  “big  picture”  thinking  (avoid  oversimplification) 

•  Promote  shared  understanding  of  problems  (build  consensus) 

•  Find  interventions  to  break  out  of  ongoing  dynamics  (recovery) 

•  Avoid  future  counter-productive  behaviors  (prevention) 
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Anatomy  of  an  Archetype: 
Causal  Loop  Diagrams 


Depict  qualitative  “influencing”  relationships  (increasing  or  decreasing) 
and  time  delays  between  key  variables  that  describe  the  system 

Show  relationship  direction  by  labeling  them  Same  (+)  or  Opposite  (-) 
to  indicate  how  one  variable  behaves  based  on  the  previous  variable 

Consist  primarily  of  two  types  of  feedback  loops: 


•  Balancing  -  Changes  to  variables  alternate,  achieving  equilibrium 

•  Reinforcing  -  Changes  to  variables  reinforce,  moving  in  one  direction 
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“Firefighting”  -  Acquisition  Archetype 
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If  design  problems  in  the  current 
release  are  higher  than  the  tolerance 
for  them,  more  resources  must  be 
dedicated  to  fix  them.  This  reduces 
problems,  but  now  fewer  resources 
can  work  on  the  next  release.  This 
undermines  early  development 
activities  which,  after  a  delay, 
increases  the  number  of  design 
problems  in  the  next  release. 
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“Sacrificing  Quality”  -  Acquisition  Archetype 
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As  schedule  pressure 
increases,  processes  are 
shortcut,  quality  suffers,  and 
errors  increase — requiring 
more  re-work.  However,  re¬ 
work  consumes  resources, 
which  increases  schedule 
pressure,  and  the  cycle 
repeats  and  worsens. 
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Acquisition  Archetypes 


There  are  many  recurring  patterns  of  behavior  in  software  acquisition 
and  development  that  have  been  modeled  using  Systems  Archetypes 
and  CLDs: 


Sacrificing  Quality 
Firefighting 

The  “Bow  Wave”  Effect 
Underbidding  the  Contract 
Shooting  the  Messenger 
Robbing  Peter  to  Pay  Paul 
Longer  Begets  Bigger 


The  90%  Syndrome 
Requirements  Scope  Creep 
Feeding  the  Sacred  Cow 
Brooks’  Law 

PMO  vs.  Contractor  Hostility 
Staff  Burnout  and  Turnover 
The  Improvement  Paradox 
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Next  Steps  and  Future  Directions 


Pattern  Library  of  Acquisition  Archetypes 

•  Eleven  Acquisition  Archetypes  have  been  described 

•  Plan  to  identify  additional  acquisition  dynamics  &  root  causes 

Collaborative  Consulting 

•  Help  customers  identify  program-specific,  counter-productive 
behaviors 

Learning  Experiments 

•  Interactive  “hands-on”  exercises  that  demonstrate  key  dynamics  in 
software  acquisition  programs 

Acquisition  Archetypes  Workshop  (2  days) 

•  “Improving  Acquisition  Practice  and  Avoiding  Patterns  of  Failure” 
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Systems  Engineering  Effectiveness  Survey 
(2004  -  2007) 


Hypothesis:  The  effective  performance  of  SE  best  practices  on  a 
development  program  yields  quantifiable  improvements  in  the  program 
execution  (e.g.,  improved  cost  performance,  schedule  performance, 
technical  performance). 

Objectives: 

•Characterize  effective  SE  practices 

•Correlate  SE  practices  with  measures 
of  program  performance 

Approach: 

•Distribute  survey  to  NDIA  companies 
•SEI  analysis  and  correlation  of  responses 

Survey  Areas: 

Process  definition  Trade  studies 

Project  planning  Interfaces 

Risk  management  Product  structure 


Project  reviews 
Validation 

Configuration  mgmt 


Requirements  development  Product  integration  Metrics 

Requirements  management  Test  and  verification 
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Total  SE  Capability  (SEC)  vs 
Project  Performance  (Perf) 


1.00- 


0.75- 

0.50- 

0.25- 

0.00- 


15% 

12% 

56% 

46% 

59  % 

39% 

13% 

29% 

31  % 

Lower 
Capability 
(x  £2.5) 
N  =  13 


Moderate 
Capability 
(2.5<x<3.0) 
N  =  17 


Higher 
Capability 
(x  £3.0) 

N  =  16 


Best 

Performance 
(x  >  3.0) 

Moderate 
Performance 
(2.5  <  x  £  3.0) 

Lower 

Performance 
(x  <  2.5) 


Gamma  =  0.32 
p  =  0.04 


Projects  with  better  Systems  Engineering  Capabilities  deliver  better  Project  Performance 

(cost,  schedule,  functionality) 
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Results:  Summary  of  Relationships 

http://www.sei.cmu.edu/publications/documents/07.reports/07sr014.html 


Relationship  of  SE  Processes  to  Program  Performance 
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ASP:  Summary 


The  SEI,  through  the  Acquisition  Support  Program,  works  directly 
with  key  acquisition  programs  to  help  them  meet  their  objectives. 


ASP  looks  for  common  themes  and  solutions  and  packages  them 
for  wider  dissemination  and  use. 
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For  More  Information 


Brian  Gallagher 

Director,  Acquisition  Support  Program 
Software  Engineering  Institute 
4500  Fifth  Ave. 

Pittsburgh,  PA  15213-3890 
(412)  268-7157 
bg@sei.cmu.edu 


Software  Engineering  Institute 


Army 

Cecilia  Albert,  cca(p).sei.  cmu.edu 


Navy 

Rick  Barbour,  reb(a).sei.  emu,  edu 

Air  Force 

John  Foreman,  itf(a)_sei.cmu.edu 

Intelligence  Community 

Rita  Creel,  rc(a)_sei.  cmu.edu 

Civil  Agencies 

Steve  Palmquist,  msp(q).sei.  emu,  edu 

Knowledge  Integration  &  Transfer 

Linda  Levine,  IK&.sei.  cmu.edu 


http:/ '/www.  sei.  emu,  edu/proqrams/acciuisition-support 
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